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METHOD AND APPARATUS FOR PERFORMING 
BIGITAL-TO-BIGITAL VIDEO INSERTION 

CROSS-REFERENCES TO RELATED APPLICATIONS 
5 This application claims the benefit of U.S. provisional Application Serial No. 

60/161,320, entitled "METHOD AND APPARATUS FOR PERFORMING DIGITAL-TO- 
DIGITAL VIDEO ADVERTISEMENT INSERTION," filed October 25, 1999, which is 
incorporated herein by reference in its entirety for all purposes. 



1 0 BACKGROUND OF THE INVENTION 

The invention relates generally to digital video processing and more 
specifically to a method and apparatus for inserting a compressed digital advertisement 
video into a compressed digital program video. 

In modem television programming, it is desirable to seamlessly insert 

15 advertising information into a program video such that a viewer does not observe unsightly 
artifacts or other anomalies on a television screen when the program transitions into an 
advertisement or vice versa. The process by which advertisement is spliced into a program 
video is known as "ad insertion' 1 . Conventionally, ad insertion is typically achieved by 
merging an analog advertisement video into an analog program video (which is also known 

20 as a "digital-into-analog" process). This insertion technique works by switching between 
(advertisement and program) video sources and broadcasting the multiplexed video to the 
viewer. If the advertisement is stored in a digital format, it is typically converted into an 
analog advertisement video prior to the ad insertion. Frame synchronization is typically a 
major concern in such an analog-based system, and other issues are not quite as critical. 

25 Since the video signal being broadcast is analog, the ad insertion equipment produces the 
multiplexed video in analog format regardless of whether the advertisement is stored in 
analog or digital format. 

As television transmission systems switch to compressed digital formats, the 
need to insert advertisement using a "digital-into-digital" process becomes obvious. The 

30 digital-into-digital process achieves ad insertion by switching between compressed 

advertisement and program video streams and broadcasting the multiplexed output video 
stream to the viewer. Conventional solutions for the digital-into-digital process, however, 
are fraught with problems since video compression schemes such as the ones defined by the 
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Moving Pictures Expert Group (MPEG) standard focuses mainly on the relationships 
between frames of video and packets of audio. 

Splicing in the compressed digital domain to achieve ad insertion generally 
cannot be accomplished with the same degree of freedom as in the analog domain or in the 
5 uncompressed digital domain. The profile (or characteristics, which typically include the bit 
rate) of a compressed video stream continually changes over time depending on the contents 
of the video, the parameter values used to encode the video, and other factors. Unless care is 
taken to ensure that the splicing is accomplished at the proper points within the compressed 
video stream and with matched encoding profiles for the compressed streams to be merged, 

10 various unsightly artifacts or other anomalies can result in the multiplexed output video. 
Such artifacts and anomalies may include, for example, severe macroblocking, dropped 
frames, macroblock panics, audio glitches, pop gargling noises, drop outs, and so on. 

Therefore, digital-into-digital ad insertion techniques that seamlessly merge 
compressed digital advertisement video into compressed digital program video are highly 

15 desirable. 

SUMMARY OF THE INVENTION 
The invention provides ad insertion techniques that overcome disadvantages 
normally associated with conventional ad insertion techniques and can be used to seamlessly 

20 merge compressed digital advertisement video into compressed digital program video. The 
techniques of the invention utilize program video "profiling" to ensure that the digitally 
encoded advertisement video (i.e., compressed advertisement video) has a profile that is 
similar to that of the compressed digital program video at approximately the time the 
compressed advertisement video is inserted into the compressed program video. By using 

25 the techniques of the invention, the video decoder (e.g., in the subscriber equipment) can 
seamlessly decode both the program and advertisement without producing visible artifacts 
on the display during or after the transition from program to advertisement or advertisement 
to program. 

The techniques of the invention can be advantageously applied in a digital 
30 information distribution system such as a video on demand system or a server-centric 

interactive program guide system. Such information distribution system may include service 
provider equipment, a data distribution network, and subscriber equipment that typically 
includes a video decoder and a display. 
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An embodiment of the invention provides an ad insertion processor (which 
typically resides in the service provider equipment) capable of seamlessly inserting a second 
compressed (e.g., advertisement) video stream into a first compressed (e.g., program) video 
stream. The ad insertion processor includes a real time encoder (e.g., an MPEG encoder), a 
profiler, and a multiplexer, and typically further includes a buffer. To process a number of 
program and advertisement videos, the ad insertion processor is typically implemented with 
a number of real time encoders, buffers, profilers, and multiplexers. 

The profiler receives the compressed program video stream and provides a 
profile for this stream. The profile is typically in the form of a continuous information 
stream that includes bit rate information (e.g., a high and low bit rate value and a mean bit 
rate value) and may further include VBV (video buffering verifier) buffer information used 
for the compressed video stream. The profiling information for the compressed program 
video stream is used to facilitate the splicing of the compressed advertisement video stream 
into the compressed program video stream in a seamless manner. 

An advertisement video source provides a number of advertisement videos. 
Upon receiving a control signal (e.g., a "pre-roll" signal that typically occurs six seconds 
before the advertisement is needed), the ad insertion source provides a selected 
advertisement video to the real time encoder. 

The real time encoder receives and encodes the advertisement video in 
accordance with a particular encoding scheme to generate the compressed advertisement 
video stream. The real time encoder further controls the encoding of the advertisement 
video based at least in part on the profile of the compressed program video stream such that 
the profile of the compressed advertisement video stream approximately matches the profile 
of the compressed program video stream at the point in time the advertisement video is 
inserted into the program video. 

The buffer may be utilized to buffer the compressed advertisement until it is 
needed by the multiplexer (e.g., to buffer six seconds of video). The buffer can provide the 
compressed advertisement video to the multiplexer upon receiving another control signal 
(e.g., a "roll" signal). 

The multiplexer receives the compressed advertisement and program video 
streams and inserts the compressed advertisement video stream into the compressed program 
video stream. This can be achieved by switching from the compressed program video 
stream to the compressed advertisement video stream, splicing the streams together, and 
providing the spliced stream as an output video stream. By using feedback of the profile of 
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the compressed program video to control the encoding of the advertisement video, when the 
multiplexer switches from the compressed program video to the compressed advertisement 
video, the switching appears seamless at a decoder. 

The profiler further collects information regarding the profile of the 
5 compressed advertisement video such that the encoding parameters for the advertisement can 
be adjusted to ensure that the profile of the compressed advertisement video is similar to that 
of the compressed program video at the time the program video was interrupted to insert the 
advertisement video. In this manner, the transition back to the compressed program video 
from the compressed advertisement video will also be seamless. 
10 The invention further provides other systems and methods that implement, 

process, and/or facilitate the ad insertion techniques described herein. 

The foregoing, together with other aspects of this invention, will become 
more apparent when referring to the following specification, claims, and accompanying 
drawings. 

15 

BRIEF DESCRIPTION OF THE DRAWINGS 

The teachings of the invention can be readily understood by considering the 
following detailed description in conjunction with the accompanying drawings. 

FIG. 1 depicts a block diagram of a digital information distribution system 
20 capable of implementing various aspects of the invention; 

FIG. 2 depicts a block diagram of an embodiment of an ad insertion processor 
in accordance with an embodiment of the invention; 

FIG. 3 is a more detailed block diagram of an embodiment of a portion of the 
ad insertion processor of FIG. 2, which can be used to splice an advertisement video into a 
25 program video; and 

FIG. 4 is a flow diagram of a video insertion process in accordance with an 
embodiment of the invention. 

DESCRIPTION OF THE SPECIFIC EMBODIMENTS 

30 FIG. 1 depicts a block diagram of a digital information distribution system 

100 capable of implementing various aspects of the invention. System 100 includes service 
provider equipment 102, a data distribution network 104, and subscriber equipment 106. In 
an embodiment, service provider equipment 102 includes a digital program video source 108 
(which may be implemented with a video server), a controller 1 10, an ad insertion processor 
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112, and a modulator 114 (which may be implemented as a quadrature amplitude modulation 
(QAM) modulator). Service provider equipment 102 couples to data distribution network 
104, which may be a hybrid fiber-coax network. Subscriber equipment 106 also couples to 
network 104 and includes a video decoder 116 (e.g., an MPEG decoder) and a display device 
5 118 (e.g., a television set). 

Digital program video source 108 provides a number of streams of 
compressed digital program video to ad insertion processor 112. As used herein, the term 
"compressed digital program video" or "compressed video" includes the associated digital 
audio information, which may or may not be compressed. The compressed program video 
10 streams from source 108 maybe MPEG-compliant streams. However, streams produced 

using other encoding schemes can also be processed in accordance with the techniques of the 
invention. 

Ad insertion processor 112 generates a number of compressed advertisement 
video streams, inserts the compressed advertisement video streams into the compressed 

15 program video streams, and provides one or more output video streams to modulator 114. 
Modulator 114 then modulates the output video streams onto one or more carriers, and the 
modulated signals are transmitted via data distribution network 104 to subscriber equipment 
106. Subscriber equipment 106 demodulates the signal, decodes a selected video stream 
using video decoder 116, and provides the decoded video to display 118. 

20 Ad insertion processor 112 provides a seamless transition between the 

compressed program video that is distributed to the subscribers and the compressed 
advertisement video that is inserted into the compressed program video, and is described in 
further detail below. Controller 110 provides the control signals for the insertion of the 
compressed advertisement video into the compressed program video. 

25 FIG. 2 depicts a block diagram of an embodiment of ad insertion processor 

112. In this embodiment, ad insertion processor 112 includes an advertisement video source 
200, a number of real time (one or more) encoders 202, a number (one or more) of buffers 
204, and a processing unit 206. In an embodiment, processing unit 206 includes a profiler, a 
splicer, and a multiplexer. 

30 Advertisement video source 200 provides the advertisements to be inserted 

into the program videos. Advertisement video source 200 may be implemented with a server 
coupled to a bulk storage device (e.g., a disk drive, magneto-optical drive, or some other 
storage device) used to store a large number of advertisements that can be selectively 
retrieved. The stored advertisements may be in a particular format and may conform to, for 
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example, the CCIR-601 video standard. Advertisement video source 200 receives from 

controller 1 10 control information identifying when an advertisement should be inserted into 

the compressed program video stream, and uses the control information to initiate the 

processing (e.g., encoding) of the advertisement. At the designated times, the selected 
5 advertisements are streamed out of advertisement video source 200 to the assigned real time 

encoders 202 on a frame-by-frame or sub-frame (slice) basis. 

Each real time encoder 202 may be assigned to encode a particular 

advertisement video from advertisement video source 200 or any number of encoders (e.g. 

two) can be used to encode the advertisement videos where each advertisement video time- 
10 shares the available encoders. Real time encoders 202 then provide the compressed 

advertisement video streams to respective buffers 204, which buffer the compressed streams 

until they are needed. 

Processing unit 206 receives a number of compressed program video streams 

(e.g., from program video source 108). For each compressed program video stream to be 
1 5 inserted with advertisements, the profiler within processing unit 206 provides a profile for 

the compressed stream. Processing unit 206 further receives compressed advertisement 

video streams from buffers 204 and inserts the advertisements into the proper program 

videos. 

FIG. 3 is a more detailed block diagram of an embodiment of a portion of ad 
20 insertion processor 112, which is used to splice an advertisement video into a compressed 
program video stream. As shown in FIG. 3, the compressed program video is provided to a 
profiler 310 and a multiplexer 312 within processing unit 206. 

Profiler 310 provides information regarding the profile of the compressed 
program video stream. This profile typically comprises information pertaining to various 
25 parameters associated with the compressed stream such as, for example, bit rate information, 
VBV (video buffering verifier) buffer information that is typically used for MPEG encoding, 
intra DC coefficient precision, group of pictures (GOP) prediction format, and so on. The 
bit rate information may include, for example, a high bit rate value, a low bit rate value, and 
a mean bit rate value for the compressed stream. As used herein, the term "profile" includes 
30 the context represented by "levels" in MPEG standard and it is not bound to the defined 

profile & level definitions and combinations used in MPEG standard. In general, a number 
of parameters are selected as representative of the program video stream encoding scheme 
and these parameters are continuously tracked by the profiler 310 for any instant parameter 
changes in the program bitstream. The profile information for the compressed program 
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video is provided (i.e., fed back) to a real time encoder 202x assigned to encode the 
advertisement video to be inserted into that program video. 

As shown in FIG. 3, real time encoder 202x encodes the advertisement video 
in accordance with a particular encoding scheme to generate the compressed advertisement 
5 video stream. Real time encoder 202x may be implemented as an MPEG-2 encoder that 
generates an MPEG-compliant stream. Alternatively, real time encoder 202 may be 
designed to encode the advertisement video using some other encoding scheme. Typically, 
real time encoder 202x implements the same encoding scheme used to generate the 
compressed program video stream. 

10 The encoding of the advertisement can begin a particular "pre-roll" time 

period (e.g., six seconds) prior to the time the advertisement is to be inserted into the 
compressed program video. The encoding start time can be indicated by a "pre-roll" signal 
that is received by controller 110 and which indicates that the advertisement will be needed 
in approximately that pre-roll time period (e.g., in six seconds). Controller 110 then notifies 

15 ad insertion processor 1 12 to produce the compressed advertisement video within that pre- 
roll time period. Thus, upon receiving the pre-roll signal, advertisement video source 200 
can begin streaming frames or slices of the advertisement video to the assigned real time 
encoder 202x, which compresses the advertisement. 

The encoding is typically initiated a short time (e.g., the pre-roll time) prior to 

20 the time at which the compressed video stream will be needed. This allows the real time 
encoder a sufficient amount of time to adjust the encoding parameters and then encode the 
advertising content. 

The compressed advertisement video stream is then provided to an associated 
buffer 204x, which buffers the compressed advertisement video for that pre-roll time period 
25 (e.g., six seconds). If real time encoder 202x is capable of rapidly encoding video, buffer 
204x may not be needed and, in that case, encoder 202x may be coupled directly to 
multiplexer 312 within processing unit 206. 

In an implementation, a "roll" signal by controller 110 informs that the 
advertisement is to be inserted into the program video in a particular roll time period (e.g., 
30 one second). If buffer 204x is not employed, encoder 202x can begin encoding the 

advertisement video at this time and directly provide the compressed advertisement video 
stream to multiplexer 312. 

At the specific point in time the compressed advertisement video is to be 
inserted into the compressed program video, multiplexer 312 switches from the compressed 
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program video stream to the compressed advertisement video stream. Splicer 314 then 
splices the compressed advertisement video stream to the compressed program video stream 
to generate a compressed output video stream. 

Prior to the time the advertisement video is inserted into the compressed 
program video stream, the profiling information that characterizes the compressed program 
video is provided (i.e., fed back) to real time encoder 202x used to encode the advertisement 
video for this program video. The profile is then used to adjust the encoder parameters to 
ensure that the profile of the compressed advertisement video approximately matches the 
profile of the compressed program video at the point when multiplexer 312 switches stream 
and the compressed advertisement video is inserted into the compressed program video. 
Since the advertisement video has been compressed to ensure that the profile of the 
compressed advertisement video stream is similar to that of the compressed program video 
stream, the insertion of the advertisement into the program video appears seamless. 

Various encoding parameters for the advertisement video can be adjusted 
based on the profile of the compressed program video. For example, the quantization step 
size used in the encoding process can be adjusted to achieve the desired profile. The 
quantization step size can be decreased if a higher bit rate is needed and can be increased if a 
lower bit rate is required. The VBV buffer information can also be used in the encoding 
process to ensure that the multiplexed output video stream will not cause buffer overflow or 
underflow at the decoder. These adjustments of the encoding process are known in the art 
and not described in further detail herein. 

The contents of the program video may be very different from that of the 
advertisement video and this may result in a difference in picture quality (as measured by the 
signal-to-noise ratio) for the same profile. In an embodiment, for a particular period of time 
(e.g., several seconds) near a splicing point, the quality of the advertisement video can be 
matched to that of the program video. This matching of the picture quality of the two videos 
to be spliced can be achieved by adjusting the profile parameters. 

In an embodiment, while the compressed advertisement video is streamed to 
multiplexer 312 (after the advertisement video has been spliced to the program video 
stream), profiler 310 collects profile parameter information according to which the encoding 
parameters are adjusted. Profile parameters such as the instantaneous bit rate of the 
advertising video stream are adjusted so that the profile parameter values closely matches 
program video parameters at the time the program video was interrupted. This is particularly 
important at the time the advertising video ends and is transitioned back to program video. 

8 
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The use of the profile to continuously adjust the encoding parameters of the advertisement 
video ensures that the splicing of the program video back into the advertisement appears 
seamless. 

A number of advertisements may be inserted sequentially into the program 
5 video using the techniques described above. In an embodiment, the advertisements may be 
encoded one after another using a number of real time encoders 202 that may be arranged in 
a cascade configuration. Each advertisement may be encoded based on the profile obtained 
for the program video into which the advertisement is to be inserted. Multiplexer 312 can 
switch from one encoded advertisement video stream to another until all advertisements 

10 have been inserted. At the end of the advertisements, multiplexer 312 switches back to the 
compressed program video stream. The profile of the compressed program video is used to 
adjust the encoder parameters for the advertisements and to ensure that the last 
advertisement to be encoded has a profile that approximately matches the profile of the 
compressed program video at the point in time the program video was paused to insert the 

15 first advertisement. In this manner, the advertisement videos can be seamlessly inserted into 
the program video. In addition to the last advertisement video, the first advertisement video 
may also be encoded with the same profile as the compressed program video to provide 
seamless transition. 

In an alternative embodiment, the advertisements may be encoded one after 

20 another using only one real time encoder 202. Again, the encoding of each advertisement 

can be controlled based on the profile of the compressed program video to ensure a seamless 
transition between the advertisements and the program video. 

Because the profile of the compressed program video stream was used to 
encode the advertisements inserted into the program video, the decoder within the subscriber 

25 equipment is able to properly decode the multiplexed output video stream without producing 
artifacts and anomalies on the video display. Since the profile of the inserted advertisement 
is similar to that of the program video, the decoding parameters and the decoder buffer are 
well behaved (e.g., no buffer overflow). Thus, the decoder can seamlessly transition from 
decoding the compressed program video stream to decoding the compressed advertisement 

30 video stream. 

In the above description, the program video is received in compressed form 
and the advertisement video is encoded based on the profile of the compressed program 
video. In other embodiments, the program video can also be received in uncompressed 
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form. In this case, the program video can be encoded in the normal manner and the profile 
for the advertising video can be generated as described above. 



advertisement video into program video. The techniques described herein can also be 
5 adapted for encoding and inserting (i.e., splicing) any types of video or other source 

materials (e.g., audio, and so on). Moreover, the techniques described herein can be used to 
splice together compressed streams for portions of frames of videos (e.g., to splice a first 
compressed stream for the upper half of the frame with a second compressed stream for the 
lower half of the frame). 

1 0 FIG. 4 is a flow diagram of a video insertion process 400 in accordance with 

an embodiment of the invention. Video insertion process 400 can be used to insert a second 
compressed video stream (e.g., for an advertisement) into a first compressed video stream 
(for a program video). Initially, the first compressed video stream is received, at step 412, 
and a profile for the stream is determined, at step 414. The profile can include the 

1 5 information described above. 



a particular encoding scheme and further based on the profile of the first compressed video 
stream to generate the second compressed video stream, at step 418. The encoding of the 
second video is controlled continuously based at least in part on the profile of the first 

20 compressed video stream, at step 420. The encoding can be initiated prior to the time the 
second compressed video stream is needed, and the encoding can be adjusted such that the 
profiles for the first and second compressed video streams approximately match. If the 
profiles for the streams do not match (within the requisite degree), the encoding parameters 
are further adjusted back at step 420. Otherwise, if the profiles approximately match, the 

25 second compressed video stream can be spliced and multiplexed into the first compressed 
video stream, at step 426. 



any person skilled in the art to make or use the invention. Various modifications to these 
embodiments will be readily apparent to those skilled in the art, and the generic principles 
30 defined herein may be applied to other embodiments without the use of the inventive faculty. 
Thus, the invention is not intended to be limited to the embodiments shown herein but is to 
be accorded the widest scope consistent with the principles and novel features disclosed 
herein. 



For clarity, various aspects of the invention are described for insertion of 



A second video is then received, at step 416, and encoded in accordance with 



The foregoing description of the preferred embodiments is provided to enable 
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